Force/torque sensor, apparatus and method for robot teaching and operation

ABSTRACT

This invention relates to force/torque sensor and more particularly to multi-axis force/torque sensor and the methods of use for directly teaching a task to a mechatronic manipulator. The force/torque sensor has a casing, an outer frame forming part of or connected to the casing, an inner frame forming part of or connected to the casing, a compliant member connecting the outer frame to the inner frame, and one or more measurement elements mounted in the casing for measuring compliance of the compliant member when a force or torque is applied between the outer frame and the inner frame.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/291,612 filed Mar. 4, 2019, now allowed, which, in turn, is acontinuation of U.S. patent application Ser. No. 15/600,105 filed May19, 2017, now U.S. Pat. No. 10,260,970 issued Apr. 16, 2019, which, inturn, which is a divisional of U.S. patent application Ser. No.14/802,337 filed Jul. 17, 2015, now U.S. Pat. No. 9,696,221 issued Jul.4, 2017, which, in turn, that is a continuation-in-part of InternationalApplication No. PCT/CA2014/050033 filed Jan. 17, 2014 designating theUnited States, that claims priority of U.S. provisional patentapplication 61/754,507 filed Jan. 18, 2013, the contents of which arehereby incorporated by reference.

TECHNICAL FIELD

The present relates to the field of force/torque sensors and intuitiveprogramming used in robotics and other applications. More specifically,it relates to teaching a mechatronic manipulator by human gesture usingmulti-axis capacitive force/torque sensors combined with specificcontrol algorithms to allow a human operator to precisely move themanipulator end-effector at configurations to register. These sensorsare also referred to as force/torque transducers and intuitiveprogramming is also referred to as human/robot collaboration or robotteaching.

BACKGROUND

This invention relates to force/torque sensor and more particularly tomulti-axis force/torque sensor and the methods of use for directlyteaching a task to a mechatronic manipulator.

A mechatronic manipulator is a mechanical device controlledelectronically to move or orient an end-effector in space. The termmanipulator in this document will refer to mechatronic manipulatorhaving one or more degree-of-freedom (DOF). One category of mechatronicmanipulator is a robot manipulator as defined by International StandardISO 8373 being an “automatically controlled, reprogrammable multipurposemanipulator programmable in three or more axes”.

Manipulators can be used to execute various processes such as arcwelding, spot welding, painting, pick and place, assembly, dispensing,polishing or deburring, just to name a few. Each process usually makeuse of specific type of end-of-arm-tool (EOAT) installed at the robotend-of-arm, also called the robot wrist or face-plate.

Programming a robot typically consists in the three following aspects.

-   -   1. Recording various points (positions and/or orientations of        the EOAT in space)    -   2. Adding process parameters: Where the process should start and        stop, how the EOAT should be controlled in the sequence, etc.    -   3. Logic programming, detailing in the program what the robot        should do in different circumstances. This logic programming can        use information that is internal to the robot and its controller        as well as sensor or information from external device(s).

The most widely used robot programming methods will fall into one of thefollowing categories.

1. Using a teach pendant.

-   -   A teach pendant is a terminal linked to the robot controller. It        has a screen as well as several buttons to display and input        information. Using the teach pendant buttons, the user jogs the        robot to different positions to be recorded, adding process        information manually or using pre-defined parameters database.        Logic is also added using the robot programming interface on the        teach pendant. For very complex tasks, code can be written in        the robot specific language or on a computer to later be        imported in the teach pendant.

2. Self-programming from sensor data.

-   -   In this case, the robot will be pre-programmed by experts and        the software will be able to make use of sensor data to generate        automatically robot path (position sequence), process parameters        and logic for a given task.

3. Off-line programming.

-   -   In this case, the robot and its surroundings are simulated in a        virtual environment. Starting from 3D drawings, a software will        generate automatically or with little user aid the robot path        (position sequence), process parameters and logic. The program        is then transferred from the simulation environment to the        robot.

Current robot programming methods present various challenges to therobot users, mainly because they require a good knowledge of robotkinematics, robot programming language and software. Processes are wellknown by people in the factories within different industries, theend-users. On the other hand, robots are just a tool for production andhow to use them is not necessarily well known by the end-users.Programming robot thus involves a supplemental skill-set that needs tobe learned and is also very time consuming, whatever programming methodis involved. Using a teach pendant is the simplest but can be very timeconsuming. Still, the user needs to learn how to program robot on top ofknowing the process. Self-programming from sensor data is fast duringoperations, but implies very complex programming up-front and can onlybe done initially by advanced robot users. Off-line programming is veryefficient at generating complex paths from 3D part drawings and robotmodels. The differences between the simulated world and the real worldneed calibration and touch-ups after the generated program has beenimported. The user in this case does not only have to learn how toprogram a robot but also need to learn the off-line programmingsoftware, which also implies knowing complex robotic notions.

To cope with the current programming challenges, new robots have beenbrought to market:

-   -   Universal Robots (http://www.universal-robots.com/)    -   Rethink Robotics Baxter robot (http://www.rethinkrobotics.com/)    -   Kuka Light Weight Robots        (http://www.kuka-robotics.com/en/products/addons/lwr/)    -   ABB Frida    -   Industrial robots:        -   Assist device U.S. Pat. No. 7,120,508

These robots have been specifically designed with the appropriatesensing, mechanical properties and software to allow teaching a task bydemonstration. These robots are inherently safe to physically interactwith humans during programming. This enables new ways of programmingthat is closer to teaching in the sense that no code is typed by theuser and no position jogging using the teach pendant buttons are used,mainly motion is shown directly to the robot. The manipulator isdirectly led in space by the user that holds it directly to record thepath. This approach simplifies the user learning. How these new robotsare built to be inherently safe has the drawback of making themunsuitable for several processes requiring higher stiffness,repeatability, precision, payload, etc.

The current invention aims at reducing programming time and complexityof programming robots and other manipulators of any types, not only thenew robots with inherently safe programming human-robot interaction. Tothis end, a force-torque sensor is installed to an existing robot.Combined with a specific software, this device can interpret the user'sintent at moving a robot by holding it. Points and/or paths can then berecorded easily and quickly. Process and logic are also programmed usinga simplified user interface on the sensor or elsewhere on the system. Tomake the solution viable, the sensor need to be cost-effective, robust,present no drift, output a clean signal at a high frequency rate andeasy to install in conjunction with the different EOATs and on thedifferent manipulators.

Many multi-axis force sensors have been proposed in the last 30 years. Afirst attempt to build a multi-axis forces and torques sensor is the“Device for measuring components of force and moment in pluraldirections” disclosed in U.S. Pat. No. 4,448,083 (publication date: 15May 1984). In the latter they proposed to measure forces and momentsalong 3 orthogonal axis using a rigid structure on which is fixed straingauges to measure local distortion. A variation to this method is theinvention disclosed in U.S. Pat. No. 4,763,531 (publication date: 16Aug. 1988) and named “Force-torque sensor”. This sensor, made formeasuring 6 forces and torques component in the cartesian space, isformed with two identically designed, integral spoke wheels. Each ofthese wheels consist in a rigid cylindrical outer ring and a rigidcenter hub connected between each other by at least three spokesdisposed in a plane. They used a total of 20 strain gauges mounted ontheses wheels to provide the 6 components of the force/moment. Theyclaimed that this invention has the advantage to be mounted with exactpositioning at very low manufacturing cost. However, while it is animprovement over the first presented approach, the amount of singlesensing element required is still very important. More recently, Kang,Dae Im et al. presented the “6-component load cell” in U.S. Pat. No.5,889,214 (publication date: 30 Mar. 1999) in which the deformablestructure consists of a cross beam having horizontal and vertical partscrossing at right angles. The force measurement is made via many straingauges attached to the bottom and side surfaces of this cross beam. Theprecision of this sensor is very high with a maximum interference error2% in the measuring of force and moment components.

All the above mentioned approaches toward building multiple axis forceand moment sensors based their measurements on strain gauges. Althoughthis has the advantage of leading to a very compact and very stiffmulti-axis force sensor, this sensing approach also has the drawback ofbeing very sensitive to noise. This is a major inconvenient particularlyfor use in robotics since most industrial robots generate a lot ofelectromagnetic noise from the high frequency digital signals used tocontrol their motors. Beside this undesirable noisy characteristic, thestrain gauge approach also suffer from the problem of drift of theirsignal over time. While noise can be filtered, drift is very hard tocompensate and in a situation where robot directly react to the level ofmeasured force, drift of the zero force point represented an issue forsafety consideration.

In reaction to these drawbacks Hirose has proposed a decade ago to usealternative way to measure forces based on optical sensors as anindication of the displacement of a given compliant structure. Thisoptic based force sensor presented a good immunity to noise, low driftover time and are relatively cheap. However, it is not very accurate formeasurement in the nano or low micro scale. Therefore, for a given forcemeasurement range, they must be coupled with a structure of an highercompliance than with strain gauge, a characteristic that can make theresulting sensor less appropriate for some application that requirestiffness and precision. An example of a multi axis force and momentsensor based on this approach is the “optical displacement sensor andexternal force detecting device” disclosed in U.S. Pat. No. 7,220,958(publication date: 22 May 2007), used three light beams, each projectedon a dual photoreceptor. The used of the photodiode technology for forcemeasurement appears to be a very promising avenue, but at this momentstill suffer from the short life of photodiode and more, from theconstant decrease of its emitting power over its effective life and fromthe higher structural compliance required in the sensor

One way to circumvent the noise sensitivity of strain gauge basedmulti-axis force sensor while keeping the sensitivity to very smalldisplacement in the structure is to use the well known relation betweenthe distance and overlap area of two conductive plates and the resultingcapacitance measurement.

The proposed invention uses a plurality of sensitive elements for whicha conductive plate is positioned on a fixed frame and another one ispositioned on a moving frame. The two frames are linked by a compliantelement such that the efforts applied on the moving frame will modifythe distance between each pair of conductive plates. The positioning ofthe sensitive elements is completely dissociated from the compliantelement. This method has several advantages compared to the existingforce/torque sensors (Wacoh [_(2004, 2006, 2004)][U.S. Pat. Nos.6,915,709 B2, 6,915,709, 6,915,709], _(Beyeler) [US 2009/0007668],_(Honda) [U.S. Pat. No. 7,757,571]). For instance, the positioning ofthe sensing elements and the design of the compliant element can beoptimized separately. As such, the sensing elements can, for example, beplaced in a single plane or in an architecture which will maximize thesensitivity, minimize the interrelation between the sensing of twodifferent elements and simplify the calibration methods. Also, becausethe compliant element is independent from the sensing elements and thusnot covered by it, its shape and material are not limited and thereforethe relation between the applied forces/torques and the resultingdisplacements can be further optimized. Finally, because the sensingelements are independent from the compliant element, they can bepre-assembled on a fixture such as a printed circuit board and later beassembled as a whole. This results in the simplification of the sensorfabrication and assembly and the reduction of its cost.

SUMMARY

This invention relates to force/torque sensor and more particularly tomulti-axis force/torque sensor and the methods of use for directlyteaching a task to a mechatronic manipulator.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood by way of the following detaileddescription of embodiments of the invention with reference to theappended drawings, in which:

FIG. 1 is an exploded perspective view representation of theforce/torque sensor where the casing has a disc-shape, according to oneembodiment;

FIG. 2a is a perspective view representation of the force/torque sensorwhere the casing has a disc-shape, according to one embodiment;

FIG. 2b is a perspective view representation of the force/torque sensorwhere the casing has a cubic shape, according to one embodiment;

FIG. 3 shows front and perspective view representations of severalcompliant mechanism architectures of the force/torque sensors, accordingto one embodiment;

FIG. 4 is a front view representation of the force/torque sensor wherethe main directions are marked on the lids to ease the integration andthe operation, according to one embodiment;

FIG. 5 is a schematic perspective view representation of theforce/torque sensor where 8 single-axis measure elements are disposed ona plane to measure all 6-DOFs, according to one embodiment;

FIG. 6a is a schematic front view representation of the force/torquesensor showing the displacement of the casings one to each other when aforce is applied to it, according to one embodiment presented in FIG. 5;

FIG. 6b is a schematic front view representation of the force/torquesensor showing the displacement of the casings one to each other when atorque is applied to it, according to one embodiment presented in FIG.5;

FIG. 6c is a circuit diagram of an oscillator producing a pulse signalwhose frequency depends on the capacitance of the measure elements;

FIG. 6d is a schematic block diagram illustrating the conversion ofmechanical displacement into a variable frequency signal andsubsequently into a force/torque computation result;

FIG. 6e is a signal diagram illustrating discriminating a pulse countaccording the embodiment of FIG. 6 d;

FIG. 7 is a table resuming the effects on the measure elements of theforce/torque sensor when forces/torques are applied to the casings,according to one embodiment presented in FIG. 5;

FIG. 8 is a schematic perspective view representation of theforce/torque sensor where 3 single-axis measure elements and 3double-axis elements are disposed on a plane to measure all 6-DOFs,according to one embodiment;

FIG. 9 is a prior art representation of a single-axis capacitivedisplacement sensor element;

FIG. 10 is a prior art representation of a double-axis capacitivedisplacement sensor element;

FIG. 11a is a schematic perspective view representation of theforce/torque sensor where 3 single-axis measure elements and 3double-axis elements are disposed on a plane to measure all 6-DOFs,showing the frames, according to one embodiment;

FIG. 11b is a schematic front view representation of the force/torquesensor where 3 single-axis measure elements and 3 double-axis elementsare disposed on a plane to measure all 6-DOFs, showing moving frames andits measure elements, according to one embodiment;

FIG. 11c is a schematic front view representation of the force/torquesensor where 3 single-axis measure elements and 3 double-axis elementsare disposed on a plane to measure all 6-DOFs, showing fixed frames andits measure elements, according to one embodiment;

FIG. 12 is a schematic perspective view representation of the apparatusto teach a manipulator by direct contact interaction with a humanoperator, according to one embodiment;

FIG. 13 is a schematic perspective view representation of the apparatusto teach a manipulator by direct contact interaction with a humanoperator, showing the different DOFs of the apparatus, according to oneembodiment;

FIG. 14 is a schematic perspective view representation of the apparatusto teach a manipulator by direct contact interaction with a humanoperator, showing the force/torque sensor fixed between the manipulatorand the EOAT, according to one embodiment;

FIG. 15 is a schematic perspective view representation of the apparatusto teach a manipulator by direct contact interaction with a humanoperator, showing the force/torque sensor and the EOAT fixed between inparallel to the manipulator, according to one embodiment;

FIG. 16 is a flow chart diagram of a method for operating the apparatusto teach a manipulator by direct contact interaction with a humanoperator in a pick-and-place application;

FIG. 17 is a flow chart diagram of a method for controlling theapparatus used to teach a manipulator by direct contact interaction witha human operator in a pick-and-place application;

FIG. 18A is a representation of a graphical user interface for aprogramming device, the graphical user interface presenting an operationsequence interface, an action type toolbox interface, an Import/Exportinterface and a parameter interface, according to one embodiment;

FIG. 18B is a representation of a keyboard display for editing aparameter value for a parameter selected from the parameter interface,according to one embodiment;

FIG. 18C is a representation of icons that are each indicative of anaction type, according to one embodiment;

FIG. 19A is a representation of the operation sequence interface of thegraphical user interface of FIG. 18A, according to one embodiment;

FIG. 19B is a representation of a parameter configuration file that maybe imported into the programming device, according to one embodiment;

FIG. 19C is a representation of an action type configuration file thatmay be imported into the programming device, according to oneembodiment;

FIG. 20 is a data flow diagram representing the exporting of aprogrammed sequence of instructions as a manipulator native code file,according to one embodiment;

FIG. 21 is a data flow diagram representing the importing of a programcode file as a programmed sequence of instructions that may be presentedin the graphical user interface, according to one embodiment;

FIG. 22 is a block diagram representing modules for interfacing betweenthe programing device and the manipulator;

FIG. 23A is a workflow diagram representing a method of programming aset of instructions from a user input with the programming device,according to one embodiment;

FIG. 23B is a workflow diagram representing a method of programming aset of instructions from an imported code file with the programmingdevice, according to one embodiment; and

FIG. 23C is a workflow diagram representing a method of programming aset of instructions from a recorded path and a user input with theprogramming device, according to one embodiment.

DETAILED DESCRIPTION Part 1: Force/Torque Sensor

According to one part of this invention, there is presented in FIG. 1 aforce/torque sensor 100. The force/torque sensor 100 is composed of acasing 110, lids 105 and 106, measure elements optionally integratedinto printed circuit board(s) 104, processor 111, optionally mechanicaldevices, such as O-rings 105, to seal the force/torque sensor 100against liquids and dust and optionally a connector 107 to communicatethe information from the force/torque sensor 100 to another device.

A skilled person will understand that such force/torque sensor 100 couldbe designed in such a way to measure force and/or torques in one or moredirections. While some applications require to measure the forces andtorques in all directions (3 forces and 3 torques), lots of applicationsrequire only few directions. For example, cartesian manipulators couldneed a sensor 100 with 3 forces while SCARA manipulators could only needa sensor 100 with 3 forces and 1 torque.

A skilled person will understand that the connector 107 could be anymeans that would allow the force torque/sensor 100 to transferinformation to another device. It is understood that it could also bewireless.

The processor 111 could be any type of processor that is able to acquirethe information from the measure elements and do the requiredcomputations to interpret those measures into forces/torques. In theapplication of robot teaching, the processor will interpret theforce/torque into user intent and generate commands to the robot. Askilled person will understand that this processor could be integratedinto the force/torque sensor 100 itself (as shown in FIG. 1) or beseparated from it. This processor 111 could also be used to allow foradditional features to the forces sensors. Such filtering functionallyexamples are herein presented. A functionality that could be added wouldbe to block or ignore certain force/torques in specific directions orwhen the force/torque is over or below a certain amplitude. In thiscircumstance the force/torque sensor 100 would return zero force/torquein the direction that have been blocked. This kind of functionalitycould be very useful in applications where forces/torques in specificdirections are to be ignored. Such applications are robotic finishing(i.e. polishing, deburring or grinding), part insertion, contactdetection, etc. A second functionality that could be done directly withthe processor 111 is the ability to change the outputs of theforce/torque sensor 100. In standard applications, forces and torquesare the most useful information to have from the force/torque sensor 100but other options are possible. In some applications, such as roboticteaching for tasks such as pick and place, painting or welding, thespeed and the direction at which the robot has to move is more relevant.For such applications, the sensor could directly output the desiredvelocity (translational and rotational) in its coordinate frame, whichis generally the same coordinate frame as the tool. Optionally, thesensor could output the desired velocity for each of the robot axis ifthe kinematics of the robot is known. In both cases, the robot would beprogrammed to move according to the velocity input from the sensor.Another sensor output possibility could be the identification of anobject according to its mass and center of mass location. As an example,a robot programmed to pick similar objects with slight mechanicaldifferences could distinguishes the object types using the sensor whichwas configured to recognize each type.

Further presented in FIG. 1, optionally, the sensor could have one orseveral holes 109 to be able to run objects through it, such as cablesor air tubes. Alternatively, the connector 107 could be in the inside ofthe ring.

Presented in FIG. 2a are front and isometric views of the casing 110 ofthe force/torque sensor 100. This casing 110 is composed of an externalframe 102; also referred as the fixed frame, an internal frame 101; alsoreferred as the moving frame and a compliant mechanism 103 linking thetwo frames 101 and 102. The compliant mechanism 103 allow the movingframe 101 to be able to move in one or more directions when aforce/torque is applied to it and when the external frame is fixed. Themagnitude of the displacement is related to the force/torque applied tothe moving frame 101.

A skilled person will understand that the casing 110 could bemanufactured using a single part or an assembly of many. It is alsounderstood that the frames 101 and 102, and the compliant mechanism 103could be made of different materials and could be of any shapes.

The measure elements 104 of FIG. 1 are independent to the compliantmechanism 103 of the force torque sensor 100. The measure elements 104measure the displacement of the frames 101 and 102 one to each other atdifferent places and in different orientations, no matter how the linkis done between the frames 101 and 102. Hence, the measure elements 104could be any type of sensor that will measure the displacement, such as(and not limiting to) capacitive sensors, Hall effect sensors,ultrasonic sensors, optical sensors, etc. The optical sensors could bethe video cameras used in mobile phones or simply DEL sensors. Theadvantage of such optical sensors is that they are cheap and easy tointegrate. The Hall effect sensors could be simply proximity sensors;many such IC chips are available on the market. To be able to measureseveral degree-of-freedom (DOF) of the force/torque sensor 101 at thesame time, a single element that is able to measure several axis couldbe used, as well as a plurality of one-axis or multi-axis measureelements 104 in complementarity.

To increase the precision of the force/torque measurement, extra measureelements 104 (more than the number of DOFs of the force/torque sensor100) could be used.

To compensate environmental variations (such as temperature fluctuation,atmospheric pressure, humidity, etc.), calibration sensor could be addedto the force/torque sensor 101. Such calibration sensor would not beaffected by frames 101 and 102 relative motions but only byenvironmental fluctuation. Hence, under environmental variations, itwould be possible to calibrate the other sensor elements 104 using themeasure of the calibration sensors. Temperature, pressure or humiditysensors could be added to the force/torque sensor to ease thisenvironmental calibration.

To compensate many environmental variations at once, the calibrationsensor could be the measure elements themselves if they are redundant(more measure elements than strictly required for the number of measureddegrees of freedom) and positioned such that there are a combination ofreadings which cannot be obtained by applying efforts on the sensor.When a reading is measured for which no displacement of the mobile framecan correspond, a correction factor can be applied on all measuringelements. The correction factor is chosen such that the correct readingsare coherent with a motion of the mobile frame of the sensor. A skilledperson will understand that a subset of the measure elements can be usedto obtain a correction factor that can be applied to all the measureelements.

A skilled person will understand that stand-alone sensors could be addedto the force/torque sensor to increase its sensing capacity. Suchsensors could be accelerometers, pressure or temperature sensors.

A skilled person will understand that the force/torque sensor 100 couldbe of any size and shape. FIG. 2b shows an example of a force/torquesensor having a cubic shape 100 a. In this specific embodiment, there isone surface of the cube (top) that is linked to the internal frame 101 awhile the five other surfaces (sides and bottom) are linked to theexternal frame 102 a. This configuration allows multiple devices to beattached to the force/torque sensor 100 a at the same time or a singledevice to alternatively be attached in different orientations. Thiscubic configuration is also best suitable for applications where thedevices that have to be fixed to the force/torque sensor 100 a have asquared shape; such as Cartesian manipulators which are often builtusing squared extrusion members. This embodiment 100 a also includescentral squared apertures 109 a to be able to run objects through it,such as cables or air tubes.

Presented in FIG. 3 are different casing 110 architectures and morespecifically different compliant mechanisms 103. The compliant mechanism103 could be of any shape and size. Different architectures will offerdifferent dynamic motions as well as different force/torque scales forthe sensor 100. Hence in some application the force/torque sensor 100would require a good stiffness while in others, compliance would bebeneficial. Same thing for the scale of force/torque, the range willdepend on the application and the geometry and the size of the compliantmechanism could be adapted to this purpose. The geometry of thecompliant mechanism 110 also affects the dynamic motion of each axis ofthe force/torque sensor 100. It is hence understood that the stiffnessof each axis would not necessarily be the same and that thischaracteristic of the compliant mechanism 103 could be adapted dependingon the application,

Presented in FIG. 4 is a front view of a lid (105 or 106) of theforce/torque sensor. The main directions 115 could alternatively bemarked on the lids to ease the integration and the operation of theforce/torque sensor 100.

Part 1—First Embodiment

FIG. 5 shows a schematic isometric view of a first embodiment 140 of themeasure elements 104 in a force/torque sensor 100. In this embodiment140, 8 single-axis measure elements (141 to 148) are used to measure all6 degrees of freedom of a force/torque sensor 100. It is understood thatthis architecture 140 or other similar variations could be used withforce/torque sensors 100 having less DOFs. In this embodiment of theinvention, the measure elements 104 are completely independent of thecompliant mechanism 103 that links the frames 101 and 102. This way, thecompliant mechanism 103 could be of any architecture since the measureelements 104 does not measure the displacement (extension orcompression) of any part of it but the global displacement (translationsand rotations) of the frames 101 and 102 one to each other. In FIG. 5,for clarity, only the fixed frames 102 is shown and moving frame 101 andcompliant mechanism 103 is not. All 8 single-axis measure elements (141to 148) are arranged on a plane (here the x-y plane) and fixed to themoving frame 101. A first measure element 141 is on the positive side ofthe x-axis and pointing in the positive side of the z-axis. This measureelement 141 can only measure local displacements of the casings in zdirection. A second measure element 142 is on the negative side of thex-axis and pointing in the positive side of the z-axis. This measureelement 142 can only measure local displacements of the casings in zdirection. A third measure element 143 is on the positive side of they-axis and pointing in the positive side of the z-axis. This measureelement 143 can only measure local displacements of the casings in zdirection. A fourth measure element 144 is on the negative side of they-axis and pointing in the positive side of the z-axis. This measureelement 144 can only measure local displacements of the casings in zdirection. A fifth measure element 145 is on the positive side of thex-axis and pointing in the positive side of the y-axis. This measureelement 145 can only measure local displacements of the casings in ydirection. A sixth measure element 146 is on the negative side of thex-axis and pointing in the positive side of the y-axis. This measureelement 146 can only measure local displacements of the casings in ydirection. A seventh measure element 147 is on the positive side of they-axis and pointing in the positive side of the x-axis. This measureelement 147 can only measure local displacements of the casings in xdirection. An eighth measure element 148 is on the negative side of they-axis and pointing in the positive side of the x-axis. This measureelement 148 can only measure local displacements of the casings in xdirection.

Shown in FIGS. 6a and 6b are schematic front views of the sensorarchitecture 140. In these figures, only 2 measure elements 145 and 146are shown to explain the concept proposed in this embodiment of theinvention. Those measure elements 145 and 146 measure the local distancebetween the fixed frame 102 and the moving frame 101. In neutral state,so under no force and no torque, the two frames 101 and 102 should beparallel and the measure elements 145 and 146 would measure thedistances 155 and 156 respectively, which are referred as the neutralmeasure values. Those distances 155 and 156 should theoretically be thesame but because of tolerances in the fabrication or otherimperfections, they could be slightly different. To correct that, amechanical calibration could be done. The mechanical calibration of themeasurements of each measuring element when zero force/torque is appliedcan be done by placing the sensor in a steady position with no forceapplied on it. The measurements of each measuring element is then storedin a non-volatile memory of the processor. These stored values arelatter subtracted from the measurements to compensate for thefabrication errors of the sensor.

FIG. 6a shows a case where a force is applied to the moving frame 101 ofthe force sensor 100 only in the positive direction of the Z axis. Inthis case, the moving frame 101 will move away from the fixed frame 102to the position 101′. Measure elements 145 and 146 will then measurelarger distances; respectively 155′ and 156′. Since the force is onlyapplied in the Z axis direction, distance 155′ and 156′ shall be thesame (after calibration correction, if needed).

FIG. 6b shows a case where a positive torque around the Y axis isapplied to the moving frame 101 of the force sensor 100. In this case,the moving frame 101 will rotate around they axis to get the orientation101″. Measure elements 145 and 146 will then respectively measure largerdistance 155″ and shorter distance 156″. Since, in this case, only atorque is applied around the Y axis, the norm of the difference betweenneutral and final distances of each measure elements 145 and 146 shallbe the same (again after calibration correction, if needed):|155″-155|=|156″-156|

To increase the precision of the force/torque measurement, the measureelements can be embodied in an oscillating circuit as illustrated inFIG. 6c such that the output frequency varies with the displacement ofthe measure elements. The variable capacitance is integrated into acircuit comprising a resistor and an inverting Schmitt trigger. Theoutput of the inverting Schmitt trigger is connected to its inputthrough a resistor. The variable capacitance is connected to the inputof the Schmitt Trigger and to the ground. As such, the output of theSchmitt trigger is an oscillating signal for which the frequency isdetermined by the value of the resistor, the variable capacitor and thethreshold points of the Schmitt trigger. By measuring the signalfrequency, it is possible to estimate the variable capacitance which isa function of the distance between the two planes of the sensingelement.

An apparatus can then be used to count the number or falling (or rising)edges of the signal during a precise period of time, as illustrated inFIG. 6d . Furthermore, as represented in FIG. 6e , an additionalmeasurement precision can be obtained if the time is measured betweenthe last edge of the signal and the end of the acquisition period. Theadvantage of such acquisition method is that the precision is very highfor a very large variation of the measured distance without the need ofany adjustment to compensation for variation of the system. Anoscillating signal is also typically more immune to external electricalnoise.

FIG. 7 shows a summary table for the different single force/torque thatcould be applied to a 6-axis force/torque sensor and the resultingmeasures that would be detected by the different measure elements (141to 148) of FIG. 5. A plus sign in the table means that, for a positiveforce/torque along the associated axis, the measure will be larger thanits associated neutral value while a minus sign means that the measurewill be smaller than its associated neutral value. A blank cell meansthat this measure element is not involved in the given force/torquemeasurement.

A skilled person will understand that in the case where there areforces/torques in various directions at the same time, the case of eachforce/torque (summarized in FIG. 7) will need to be “summed” to get thefinal result.

This architecture 140:

-   -   eases the calibration of the force/torque sensor 100    -   allows to position the measure elements 104 in such a way that        it will optimize the stroke of the displacements in function of        the applications    -   facilitates the force and torque computation since they are        partially decoupled (as shown in FIG. 6)

Part 1—Second Embodiment

FIG. 8 shows a second embodiment 160 of the first part of the inventionwhere all the measure elements (160 to 166) have at least one measureaxis oriented in the same direction (here, in the Z-axis direction). Inthis embodiment 160, 3 single-axis measure elements (161 to 163) as wellas 3 double-axis measure elements (164 to 166) are used to measure all 6degrees of freedom of a force/torque sensor 100. It is understood thatthis architecture 160 or other similar variations could be used withforce/torque sensors 100 having less DOFs. In this embodiment of theinvention, the measure elements (161 to 166) are completely independentof the compliant mechanism 103 that links the frames 101 and 102. Thisway, the compliant mechanism 103 could be of any architecture since themeasure elements (161 to 166) does not measure the displacement(extension or compression) of any part of it but the global displacement(translations and rotations) of the frames 101 and 102 one to eachother. In FIG. 8, for clarity, only the fixed frame 102 is shown. All 6measure elements (161 to 166) are arranged on a plane (here the X-Yplane) and attached to the fixed frame 102. A first one-axis measureelement 161 is on the positive side of the Y-axis and pointing in thepositive side of the Z-axis. This measure element 161 can only measurelocal displacements of the casings in the Z direction. A second one-axismeasure element 162 is located at −120° of the measure element 161 andpointing in the positive side of the Z-axis. This measure element 162can only measure local displacements of the casings in Z direction. Athird one-axis measure element 163 is located at 120° of the measureelement 161 and pointing in the positive side of the Z-axis. Thismeasure element 163 can only measure local displacements of the casingsin Z direction. A first two-axis measure element 164 is on the negativeside of the Y-axis and its first measure axis is pointing in thepositive side of the Z-axis while the second measure axis is pointing inthe positive side of the X-axis. This measure element 163 can measurelocal displacements of the casings in Z direction as well as in the Xdirection. A second two-axis measure element 165 is located at 120° ofthe measure element 164 and its first measure axis is pointing in thepositive side of the Z-axis while the second measure axis is pointing inthe positive side of the axis 168. This measure element 165 can measurelocal displacements of the casings in Z direction as well as in thedirection of the axis 168. A third two-axis measure element 166 islocated at −120° of the measure element 164 and its first measure axisis pointing in the positive side of the Z-axis while the second measureaxis is pointing in the positive side of the axis 169. This measureelement 166 can measure local displacements of the casings in Zdirection as well as in the direction of the axis 169.

This architecture 160:

-   -   eases the calibration of the force/torque sensor 100,    -   allows to position the measure elements 104 in such a way that        it will optimize the stroke of the displacements in function of        the applications,    -   simplifies the fabrication of the force/torque sensor since all        measure elements 104 are lying on a same plane and that at least        one axis of all measure elements 104 are pointing in the same        direction.

Example of the Second Embodiment with Capacitive Sensors

Presented in FIG. 9 is an isometric view of a classical capacitivedisplacement sensor element 500. It is composed of two conductive plates501 and 502 on which a charge is applied and a dielectric material 503between the two plates 501 and 502. The capacitance C of this element500 can be expressed by:

$C = \frac{ɛ_{0}{KA}}{d}$

Where C is the capacitance, co is the known permittivity of free spaceconstant, K is the known dielectric constant of the material in the gap503, A is the conjoint area of the plates 501 and 502, and d is thedistance between the plates 501 and 502. Hence, it is possible with suchsensor to compute the displacement in z direction, that corresponds tovariation in ‘d’ in the previous equation, by measuring the capacitancevariation (difference in ‘C’) electronically.

FIG. 10 shows how to use capacitive sensors to be able to measuredisplacement in more than one direction simultaneously (multi-axismeasurement). FIG. 10 shows a sensor that is able to measure thedisplacement in two dimensions but this concept could be extended tomore dimensions. Again the sensor is composed of two conductive plates601 and 602 on which a charge is applied and a dielectric material 603between the two plates 601 and 602. The two plates 601 and 602 areoverlapped in one direction (here in the y direction) so the ConjointArea (A) will increase or decrease if the plates 601 and 602 are movingone to another in the Y direction. Note that in this specific case, therelative movement of the plates in the x direction will have no effectsince the Conjoint Area (A) will not change (for small displacements).As for the classical capacitive displacement sensor element 500 of FIG.9 the relative movement of the plates 601 and 601 one to each other inthe Z direction will also affect the capacitive value. Finally note thatthe shape of the plates could be of any shapes or size. It is possible,for example, to design the plates in such a way that a same displacementin the two directions will produce a capacitance variation of a similarscale.

FIGS. 11a, 11b and 11c show an example configuration of the secondembodiment 170 of the first part of the invention where all the measureelements (171 to 176) are capacitive sensors. In this embodiment 170, 3double-axis capacitive measure elements (174 to 176) as well as 3single-axis capacitive measure elements (171 to 173) are used to measureall 6 degrees of freedom of a force/torque sensor 100. It is understoodthat this architecture 170 or other similar variations could be usedwith force/torque sensors 100 having less DOFs. In this example of thesecond embodiment of the invention, the measure elements (171 to 176)are completely independent of the compliant mechanism 103 that links theframes 101 and 102. This way, the compliant mechanism 103 could be ofany architecture since the measure elements (171 to 176) does notmeasure the displacement (extension or compression) of any part of itbut the global displacement (translations and rotations) of the frames101 and 102 one to each other. In FIG. 11, for clarity, only the fixedframe 102 and moving frame 101 are shown. The 3 single-axis capacitivemeasure elements (171 to 173) and the 3 double-axis measure capacitiveelements (174 to 176) are arranged on a plane (here the X-Y plane) andare alternately and evenly distributed (at every 60° about the z-axis)as explained for FIG. 8.

Presented in FIGS. 11b and 11c are respectively the moving 101 and thefixed 102 frames of the sensor and their associated measure components.The 3 single-axis capacitive plates 171 a, 172 a and 173 a of the movingframe 101 of FIG. 11b are associated with the 3 single-axis capacitiveplates 171 b, 172 b and 173 b of the fixed frame 102 of FIG. 11c . The 3double-axis capacitive plates 174 a, 175 a and 176 a of the moving frame101 of FIG. 11b are associated with the 3 double-axis capacitive plates174 b, 175 b and 176 b of the fixed frame 102 of FIG. 11c . The 3double-axis measure elements plates (174 to 176) have “comb” and “fir”shapes respectively for the moving 101 and the fixed 102 frames. Thoseshapes allow the double-axis capacitive sensor elements (174 to 176) tohave similar precision in both directions as well as allowing thecapacitance of each element to be increased. It is possible to isolatethe variation of the measurements for each of the sensitive dimensionsof the double-axis capacitive sensor elements (174 to 176) using themeasurements of the single-axis capacitive sensor elements (171 to 173).This is obtained by computing the position and orientation of the planewhich contains all the sensor elements using the measurements of thesingle-axis capacitive sensor elements (171 to 173). Knowing theposition and orientation of the plane, it is possible to compute thedisplacement of the double-axis sensor elements (174 to 176) for thedirection which is common to the single-axis sensor elements (174 to176). It is finally possible to remove the effect of this displacementfor the measurement from the double-axis sensor elements (174 to 176) toisolate the effect of the displacement in the other sensitivedirections.

Part 2—Apparatus and Method to Teach a Manipulator

The second part of this invention aims at giving human-robot interactioncapabilities to manipulators, using a force/torque sensor, in order tomake them easier to program.

According to one embodiment, there is presented in FIG. 12 the apparatusto teach a manipulator by direct contact interaction with a humanoperator. This apparatus is composed of a robot 1000 (in the figure, a6-DOF robot arm), a sensor 100 (in the figure a force/torque sensor) andan EOAT 1003 (in the figure a robotic welding torch). Note that theend-of-arm 1001 is part of the robot arm 1000. For simplicity, in theother figures, only the end-of-arm 1001 is shown to represent the robotmanipulator 1000. The sensor measurements are processed to obtain theuser intent on where he wants the manipulator to move in space.

Presented in FIG. 13 is an isometric view of the apparatus. In thisembodiment, the manipulator 1000 can move in 6 dimensions (3translations and 3 rotations). A skilled person will understand that themanipulator 1000 could be of any type as long as it has at least onedegree-of-freedom.

Presented in FIG. 14 are front and isometric views of one embodiment ofthe teaching apparatus. In this embodiment, a force/torque sensor 100 isfixed between the end-of-arm 1001 of the robot and the robotic tool1003. In this embodiment, the user will teach the robot by applyingforces/torques to the robotic tool 1003 with his body 1002 (here itshand). Those forces/torques are interpreted by the force/torque sensor100 and are converted into motion of the robot 1001. In this embodiment,the force/torque sensor 100 is intended to be fixed permanently to therobot end-of-arm 1001.

Presented in FIG. 15 is an isometric view of a second embodiment of theteaching apparatus. In this embodiment the force/force sensor 100 andthe robotic tools 1003 are separately attached to the end-of-arm 1001 ofthe robot allowing the force/torque sensor to be easily removed afterthe teaching task. In this embodiment, an handle 1004 could be added tothe force/torque sensor 100 to allow the user 1002 to apply force to theforce/torque sensor 100. This handle 1004 could also be equipped withany types of user input device (buttons, touch screens, etc.) that couldbe used to program with specific tasks such as record, play, EOATsettings, logic, etc.

The sensor could be installed at other places in the kinematic chain ifnot all end-of-arm DOFs need to be positioned. If the manipulator 1000is kinematically redundant for the process, several sensors could alsobe placed at different places along the kinematic chain so the desiredconfiguration can be fully controlled by the user.

In the different hardware embodiments, an algorithm needs to be used toprocess the force/torque sensor information using a control law thatallow for a smooth, precise and intuitive robot motion guidance by theuser.

It is possible to modify the behaviour of the interaction in certaincircumstances, for example if a contact with a hard surface is detected,if the speed is above or below a certain limit or in function of whichuser is teaching the robot. It is also possible to change the behaviorof the interaction in such a way to constrain chosen movements(translations or rotations), making a 6 DOFs robot moves such as aCartesian one or rotational one for instance.

In the different hardware embodiments, it is possible to compensate forthe weight of the tool 1003 or the weight of the handle 1004 that isattached to the force/torque sensor 100. For this task, there are twoways to do it:

-   -   The robot controller sends the force/torque created by the        weight of the tool/handle for its current configuration.    -   The sensor processor computes the tool/handle orientation using        an internal 3-axis accelerometer and uses pre-calibrated        information related to the weight of the tool to calculate and        remove its effect on the force/torque measurements.

The resulting manipulator 1000 movement must comply with both processrequirements and user ergonomy. The algorithm must allow positioning ofthe EOAT that is precise enough for the given process. The movement needto be easy enough to generate so it does not induce strain to the userthat is teaching the robot.

It is possible to include safety features in the sensor itself. Forexample, any overload of the sensor could result in the output of zerofor the motion of each axis, effectively stopping the robot. Also,internal accelerometer could be used to detect any collision and reactappropriately, for example by stopping the robot or moving in theopposite direction to prevent any damage. The force/torque sensor 100should be designed to send zero motion data by default, only sendingmeaningful data if all safety criteria are met.

During the robot movement or at a specific point, the robot position canbe recorded using a user input device that can take many forms. It canbe button(s) on the traditional teach pendant or buttons on the sensoror other input devices. This user input interface can also be used toblock given manipulator DOF if desired by the user. The same inputdevice or another one can be used to input process information.Accessories can be added to end-of-arm-tool to aid the teaching (i.e.welding tip, gripper jig, etc.).

FIG. 16 shows an example of a sequence 1500 that a user would need tofollow to teach a manipulator to do a simple pick and place operationwith the apparatus explained previously. In this example, themanipulator starts at an initial position, move to a predefined positionwhere it can reach the object, grab the object, move to anotherpredefined position where it has to drop the object and finally drop theobject.

FIG. 17 shows the method 1700 used to control the robot using the sensorfeedback in order to accomplish the example described above (FIG. 16).In this method, the robot is not moving unless an enabling switch istriggered by the user. When the enabling switch is activated, the sensorperforms computations to determine the robot velocity. Sequentially, thecapacitance of each sensing element is measured. Using the sensorcalibration located in a non-volatile memory, the capacitance isconverted in displacement for each sensing element, which are in turnconverted to forces/moments applied on the sensor. The pre-calibratedweight of the tool is removed from the measured forces/torques. Theseefforts are then introduced in a virtual model representing a virtualobject moving in a virtual environment. Using the previously computedvelocity and the actual forces/torques, the output velocity of thevirtual system is updated. The robot is then controlled to make itsend-effector move with this velocity, limited for safety purposes. Theloop is then synchronized to make sure that the output velocity isupdated at a fixed frequency. However, if the user presses a specificbutton, the current position of the robot is saved in the robot memoryfor future playback.

The proposed method to control the motion of a robot has many advantagesover the use of a joystick or such device located anywhere else than onthe robot, for example on a fixed table near the robot or on the teachpendant. First, although a joystick has the advantage of allowing thecontrol of a precise and constant velocity, displacing the end-effectorof the robot directly is more precise in terms of position since theapplied force (and the resulting velocity) is reduced as the robot ismoving. Also, with the proposed approach, the resulting velocity isalways in the same direction as the applied force, which is not the casewith a fixed joystick and a rotating end-effector. This simplifies thecontrol of the robot as the user never has to think about which actioncreates which motion. Finally, the interaction is more natural if theuser is applying forces directly on the tool. For example, if the toolon the end-effector is a welding torch, the user will be able to movethe torch in the same manner as if there was no robot. This furthersimplifies the use of such system to record points and trajectories.

Part 3—Interface to Teach a Manipulator/Robot

The third part of this invention aims at giving a human-robot orhuman-manipulator interaction capabilities by providing a user-friendlyinterface for programming a robot or manipulator with ease.

It has been discovered that a user friendly human-manipulator interfaceallows a user to program, test and play-back a sequence of instructionsfor a manipulator to execute without requiring knowledge of anassociated manipulator programming code language. It has further beendiscovered that during a manipulator programming process, auser-friendly interface allows a user to provide relatively quick editsof operational instruction recordings. It has further been discoveredthat a user friendly human-manipulator interface allows a user tounderstand a meaning of a programing instruction without the use ofwords. The user-friendly interface may be part of a programming deviceas presented in FIG. 18A or part of any other suitable programmingdevice. The programming device may have various suitable shapes or formsthat may include physical or virtual button(s) that arepositioned/displayed on a traditional teach pendant or that arepositioned/displayed on a manipulator sensor or that arepositioned/displayed on any other suitable input device that maytransfer data to/from a manipulator by cable or radio waves.

The interface may be used for programming the manipulator in conjunctionwith other methods of programming the manipulator, including methodsthat consist of programming a manipulator by teaching or recording amotion. For instance, during or following the recording of a robotmovement or a robot pause, additional or complementary operationalinstructions can be programmed using the programming device. Accordingto one embodiment, the interface allows the user to program manipulatorinstructions with only a single click or only a few clicks withouthaving to input programming code. This way, when the manipulator islocated in a desired position or moving in a taught manner, the user mayeffectively and quickly input a desired instruction to includeadditional instructions, to enhance a taught motion or to provideadjustments instructions. As will be further described below, variousinstructions may be programmed according to the type of manipulator.According to one embodiment, an instruction is defined by a selectedaction type and inputted parameters.

As presented in FIG. 18A, the user programming device has a userinterface 2400 for allowing an end-user to provide instructions to themanipulator. According to one embodiment, the user interface 2400 has anoperation sequence interface 2402, a toolbox interface 2404, aparameters interface 2406 and an import/export interface 2408.

Operation Sequence Interface:

Presented in FIG. 18A, the operation sequence interface 2402 is used atvarious program development phases such as when the program is beingbuilt, tested and modified. The sequence interface may also be usedduring production play-back and give an indication of the sequenceprogress.

The operation sequence interface 2402 provides for communicating to theuser an operation sequence of instructions that is being programmed orthat has been programmed for the manipulator to execute. According toone embodiment, the operation sequence is represented as a list of iconseach depicting an instruction for performing an action. Presented inFIG. 18B are three icons applicable to a welding process: straight linemotion instruction 1800, circular path motion instruction 1801 and starta welding process instruction 1802. As can be noticed, the meaning ofeach icon is easily and readily recognizable by the end-user looking oreven glimpsing at them. Depending on the area of application, there maybe different or additional icons. The icons may have various shapes andforms that differ from the ones presented in FIG. 18B such that the usercan recognize and understand their meaning with relative ease.

In one embodiment of the operation sequence interface 2402, the iconsare color coded for an end-user to associate an icon to a particularportion of the operation sequence. For example, a welding path could behighlighted in a different color for indicating to the user thatsomething particular is happening during the associated sub-sequence. Askilled person will understand that other ways of highlighting asub-sequence may be used for informing the user that somethingparticular is happening during that sub-sequence.

In another embodiment of the operation sequence interface 2402, asuccession of instructions that the manipulator must execute isrepresented in at least three parts. As presented in FIG. 19A, a firstpart represents a current instruction 1810, a second part represents aprevious instruction 1811 and a third part represents a next instruction1812. In addition, there is presented to the end-user a progressindicator 1813 of a selected instruction position with respect to acomplete sequence of instructions length. In this FIG. 19A, the progressindicator 1813 is represented within a progress bar, however other waysof representing a sequence progress to the user may also be consideredwithout departing from the scope of the present invention.

During the program development phases, the operation sequence interface2402 allows a user to add or remove instructions from the sequence. Aspresented in FIG. 19A, a current instruction can be selected by the userin order to delete it from the sequence, modify its parameters, insert anew instruction after it or replace it by another instruction. In oneexample, a current instruction 1810 is selected by the user as shown bythe highlighted box for indicating to the user that instruction 1810 isthe current instruction after which a new instruction will be inserted.

During play-back, the operation sequence interface 2402 is used toinform the user concerning the manipulator's current position andcurrent instruction being executed. According to one embodiment, withthe operation sequence interface 2402 the user can order the robot tojump to a specific instruction by selecting a desired instruction fromthe sequence.

Toolbox Interface

The toolbox interface 2404 presents to the user the available actiontypes that may be inserted in the sequence, as depicted in FIG. 18A.

According to one embodiment, the toolbox interface 2404 presents iconsto the user, each icon depicting a specific function or action type.Without the use of words, the icons provide an intuitive way for theuser to program a sequence of instructions or action types. The user mayfind it easier to remember the function each icon represents. The usermay further identify or discern the function of each icon easily andrapidly by simply looking or even glancing at each icon. The icons mayhold additional information such as an indicator that may be a colorindicator or a grayed-out icon for informing the user that a specificaction type may not be inserted in the operation sequence or at aparticular position of the operation sequence. Additional informationmay also be presented to the user via an icon overlay.

A skilled person will understand that an icon may present to the userother types of additional information and that in various differentmanners than those mentioned above, without departing from the scope ofthe present toolbox interface 2404.

While in the toolbox interface 2404, icons are presented for depictingan action type, various other user-friendly manners of presenting anaction type may be used without departing from the scope of the presentinvention. In one example, a physical or virtual numeric keypad ispresented and to each key of the keypad is associated a predefinedaction type. In another example, physical or virtual buttons arepresented, each button having an associated word or phrase describing acorresponding action type.

According to one embodiment, the action types are of two kinds: movementspecific action types and application specific action types. Movementspecific action types are universal instructions that are common amongdifferent types of robots and across various areas of application. Forexample, such action types non-restrictively include: moving along alinear path, moving along a circular path, moving along a spline andoptimizing joint speed. Application specific action types are likely todiffer depending on the area of application and robot type. For example,in a welding application, action types may include: turn on weldingtorch, adjust flame, turn off welding torch, etc. In a grippingapplication action types may include: pick up, place down, push, pull,etc.

Parameters Interface

According to one embodiment, during a playback of the instructions, theparameter interface 2406 provides an indication of the parameters usedfor a current instruction 1810 selected from the operation sequenceinterface 2402, as presented in FIGS. 18A and 19A. During programming ofthe instructions, the parameter interface 2406 further allows the userto read, add and modify parameters associated to the current instruction1810. Also, during programming of the instructions by selecting anaction type from the toolbox interface 2404, the parameter interface2406 may allow the user to define parameters associated to a selectedaction type or to edit default parameters that are associated with theselected action type.

According to one embodiment, as presented at FIG. 18A, the parameterinterface 2406 presents a set of parameters that are associated to acurrent instruction 1810 or to a selected action type. The parameterinterface 2406 further presents a key depicting icon next each parameterthat may be edited. For modifying a value of one of the editableparameters, the user may select the corresponding key depicting icon anda corresponding parameter definition interface XXX is then displayed, aspresented in FIG. 18B. With this interface XXX, the user may edit theparameter as desired. It is understandable that a new parameter may alsobe added in the same way by presenting the same parameter definitioninterface XXX or any other suitable interface.

According to one embodiment, the parameters presented to the user are inlimited numbers and are restricted to only the parameters related to thecurrent instruction 1810 or the selected action type from the toolboxinterface 2404. Moreover, the parameters may be restricted to only anecessary subset determined for example according to a current positionof the manipulator. Such limitations and restrictions provide forenhanced interface usability.

According to one embodiment, the toolbox 2404 includes action types thatare customizable and may require a customization of parameters.According to one embodiment, an action type configuration file 1902 maybe read by the programming device in order to associate a set ofparameters to a customizable action type, as presented in FIG. 19C. Inan alternate embodiment, a parameter configuration file 1900 holds theset of parameters associated to a defined customizable action type, aspresented in FIG. 19B.

Import/Export Interface

According to one embodiment, the programmed sequence of instructions maybe exported from a first programming device and imported into anotherprogramming device or multiple other programming devices. This allowsfor multiple robots to reproduce the same instructions in parallel or toproduce a backup of the programmed sequence of instructions.

Presented in FIG. 18A, the Import/Export interface 2408 allows the userto select an Export function button for exporting the programmedsequence of instructions of the programming device. The interface 2408also allows the user to select an import function button for importing aprogrammed sequence of instructions into the programming device.

According to one embodiment, the programmed sequence of instructions isa code that is readable by the manipulator. As presented in FIG. 20,when the user selects the Export function button, the programmedsequence of instructions represented by a sequence data structure 1821is read by a code generator 1822. The code generator 1822 then convertsthe sequence into the manipulator's native code 1823. The manipulator'snative code 1823 is then exported as a program code file.

According to an alternate embodiment and as presented in FIG. 21, whenthe user selects the import function button, a user selected code fileholding manipulator's native code 1823 is parsed by a manipulatorspecific code parser 1824. The code parser 1824 parses the native code1823 and from the parsed code a sequence data structure 1821 isinstantiated such that the sequence interface 2402 may present to theend user for editing or playing-back the programmed sequence ofinstructions as imported.

Interfacing with the Manipulator

Presented in FIG. 22 are the various modules used for interfacingbetween the programming device 1820 and the manipulator platform 1830.As described above, the user uses the programming device 1820 fordefining a sequence of instructions via the operation sequence interface2402. The defined sequence of instructions is passed onto themanipulator 1830 for the manipulator to perform the sequence ofinstructions by accordingly moving a tool, launching applicationspecific events and controlling the application specific events.However, for the manipulator to perform the defined sequence ofinstructions, the sequence of instructions must be converted intomanipulator specific code language.

The manipulator specific code language although precise is relativelycomplex. Moreover, there is no universal manipulator code language thathas to date been adopted and the code language for each differentmanipulator is manufacturer dependent. Therefore, an interface softwarethat connects with the manipulator is also manufacturer dependent.Consequently, it is desirable to have an interface software that ismodular and swappable. According to one embodiment, the interfacesoftware may be easily changed according to the manipulatormanufacturer. This way, the same programming device 1820 may be usedwith a manipulator of any manufacturer.

According to one embodiment, in order to interface with the manipulator,a manipulator specific code generator 1822 converts the sequence ofinstructions into manipulator specific code language. As described abovefor exporting, the sequence of instructions is represented by thesequence data structure 1821 that is used as input for the codegenerator 1822. The code generator 1822 then converts the sequence intomanipulator specific code language.

As described above, for importing a sequence of instructions from aninput file of manipulator specific code 1823, the programming device1820 uses a manipulator specific code parser 1824 for parsing theimported code 1823. A sequence data structure 1821 is instantiatedaccording to the parsed imported code thereby allowing the programmingdevice 1820 to display user recognizable icons representing the sequenceof instructions imported for further editing or playing-back.

The action type configuration file 1902 is used by the code generator1822 to translate custom instructions that are not part of the robot'sbasic instruction set into the robot's native language. Moreover, theaction type configuration file 1902 is also used by the code parser 1824to identify instructions that are not part of the robot's basicinstruction set and to include them in the interface.

Further presented is FIG. 22, is a manipulator specific monitor 1831that communicates information from the manipulator platform 1830 and tothe programming device 1820. Its purpose is to gather informationrelating to the manipulator platform 1830, such as: a current positionof the manipulator, a current instruction being executed, an alarmstatus, etc. The format of the information gathered is normallyuniversal to manipulator platforms of different manufacturer. However,in many instances the transmission protocol differs from one manipulatorplatform 1830 to another depending on the manufacturer. Consequently, amanipulator specific monitor 1831 is implemented according to themanufacturer.

Sequence Data Structure

A sequence of instructions is stored as a sequence data structure 1821.According to one embodiment, the sequence data structure 1821 is a treedata structure in which an instruction is represented by a nodecontaining data representing associated parameters. In order to displaya list of instructions to the user, the list of instructions is producedby applying a traversal algorithm to the tree data structure. It isknown that various traversal or search algorithms may be applied. Inthis embodiment, a preorder traversal pattern is applied to generate alist of instructions out of the tree data structure and the inherenthierarchy of the tree data structure is used to organize the sequence ofinstructions. Conveniently, a subsequence of instructions can be addedto the tree data structure or be manipulated independently of the restof the sequence of instructions. For example, if two identical tubeshave to be welded on a plate, a sequence of instructions may beprogrammed for welding a first tube on the plate and the sequence ofinstructions may be copied and inserted into the tree data structure asa sub-sequence with an offset for welding a second tube on the platewith an offset.

A skilled reader will understand that the sequence data structure 1821may be defined by any other suitable means of representing suchstructure (i.e. an array, a linked list, etc.).

Manipulator Specific Code Generator

The manipulator specific code generator 1822 is adapted to receive thesequence data structure and translate it into a program code that amanipulator controller can understand, as presented in FIG. 22. Thismodule 1822 uses the sequence data structure representation and convertsit into a more complex program code representation by undoing theabstraction of the programming language. According to one embodiment,the algorithm used for performing the conversion allows to visit everynode of the tree depth-first in a preorder pattern. Thus, everyinstruction of the sequence is treated in the same order as appeared inthe manipulator's native code language.

Parser

The parser 1824 provides for abstracting the manipulator's complexprogramming language, as presented in FIG. 22. The parser 1824translates a manipulator program code into a sequence of instructionsthat the programming device will be able to display in the sequenceinterface 2402. According to one embodiment, the parser 1824 applies analgorithm for reading the program code line by line. For each parsedinstruction, the associated parameters are parsed and an associatednode—to which are added the associated parameters—is created in the treedata structure.

Monitor

The Monitor 1831 receives the manipulator's state information andgenerates a manipulator current state information for the programmingdevice, as presented in FIG. 22. The manipulator current stateinformation being indicative of the manipulator's current positionand/or instruction being executed. Such information may provideinformation to the user whether or not the manipulator has reached aninstructed target.

Workflow

According to one embodiment, the following workflow may be executed, aspresented in FIG. 23A:

-   -   1. The user inputs a desired set of instructions with the        programming device for building a sequence of instructions 2300.    -   2. For testing/executing/playing-back the sequence of        instructions, the generator converts the sequence of        instructions into the manipulator's code language 2301.    -   3. The generated code is loaded into the manipulator's memory        2302.    -   4. The user starts the manipulator playback with the programming        device, either step by step or all at once 2303.    -   5. The monitor receives the manipulator's state and the        manipulator's state is displayed on the sequence interface of        the programming device 2304.

According to an alternate embodiment, the following workflow may beexecuted, as presented in FIG. 23B:

-   -   1. The user imports an existing code file into the programming        device for building a sequence of instructions 2310.    -   2. The code is parsed by the parser and converted into a simple        sequence of instructions 2311.    -   3. The sequence of instructions and parameters is modified by        the user 2312.    -   4. For testing/executing/playing-back the sequence of        instructions, the generator converts the sequence of        instructions into the manipulator's code language 2313.    -   5. The generated code is loaded into the manipulator's memory        2314.    -   6. The user starts the manipulator playback with the programming        device, either step by step or all at once 2315.    -   7. The monitor receives the manipulator's state and the        manipulator's state is displayed on the sequence interface of        the programming device 2316.

According to another alternate embodiment, the following workflow may beexecuted, as presented in FIG. 23C:

-   -   1. The user records by teaching the manipulator a desired path        with a teach pendant or a motion sensor 2320.    -   2. The user inputs a desired set of instructions with the        programming device for building a sequence of instructions with        the taught motion 2321.    -   3. For testing/executing/playing-back the sequence of        instructions, the generator converts the sequence of        instructions into the manipulator's code language 2322.    -   4. The generated code is loaded into the manipulator's memory        2323.    -   5. The user starts the manipulator playback with the programming        device, either step by step or all at once 2324.    -   6. The monitor receives the manipulator's state and the        manipulator's state is displayed on the sequence interface of        the programming device 2325.

What is claimed is:
 1. A method of training a robotic manipulatorsystem, the method comprising: providing a force/torque sensor at ornear an end effector of the robotic manipulator system; and applyingnavigating forces to said sensor and using signals from said sensor topilot the end effector into one or more engagement positions during atraining or learn mode to define motion of the robotic manipulatorsystem during task operation, wherein said sensor is mounted betweensaid end effector and said robotic manipulator system, said navigatingforces being applied to said end effector, calculating a weight of theend effector for a current position and orientation thereof from therobotic manipulator system, and said navigation forces being determinedusing correction of signals from said force/torque sensor using saidweight.
 2. The method as claimed in claim 1, wherein signals of saidsensor when no navigation forces are applied to said end effector aresubtracted from signals of said sensor while navigating forces areapplied.
 3. The method as claimed in claim 1, wherein said manipulatoris kinematically redundant for a desired process, said providing furthercomprises providing a force/torque sensor at one or more additionallocations along a kinematic chain of said manipulator, said navigatingforces being applied to said end effector and to said additionallocations so that a desired configuration can be fully controlled by auser.
 4. The method as claimed in claim 1, wherein said sensor ismounted to said robotic manipulator system near said end effector and anavigation handle is connected to said sensor for measuring forcesbetween said handle and said robotic manipulator system.
 5. The methodas claimed in claim 1, wherein said end effector is a welding tool. 6.The method as claimed in claim 1, wherein said end effector is agripper.
 7. The method as claimed in claim 1, further comprising using atraining pendant to control said manipulator and/or said end effector,and/or to record positions of said manipulator.
 8. A method ofmanufacturing a product using a robotic manipulator system, the methodcomprising: training said robotic manipulator system as claimed in claim1; and using said trained robotic manipulator system to manufacture saidproduct.
 9. The method as claimed in claim 8, wherein said force/torquesensor is used to measure contact between said end effector and anobject in a workspace to control a motion of said manipulator system.10. The method as claimed in claim 8, wherein said training said roboticmanipulator system further comprises inputting a selected action typeand an associated parameter with a programming device by selecting saidaction type from a list of action types presented by said programmingdevice to define said motion of the robotic manipulator system accordingto the applied navigation force to said sensor.
 11. The method asclaimed in claim 10, wherein said list of action types comprises actiontypes that are individually represented by an icon.
 12. The method asclaimed in claim 10, wherein said list of action types are presented ina user interface of the programming device and said list of action typesis a restricted list of action types according to a current instructionor a current position of said manipulator and/or said end effectorand/or according to a type of said end effector.